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ABSTRACT 



The basic premise of this paper is: if current management and fiscal problems 
are to be solved in higher education, planners and decision makers must adopt 
an outcome-oriented approach to planning. In the past, most planners have 
tended to be means -oriented, focusing more attention and energy on problems 
, associated with the course of action being followed and less on the end re- 
sults to be achieved. It is argued that planners should begin focusing on 
the desired , expected , and actual outcomes of programs as they relate to the 
planning process. 

Although an outcome-oriented approach to planning should be beneficial, it is 
not without its problems. Among the problems discussed are that of reaching 
consensus on institutional goals and objectives, the lack of variance problem 
in attributing outcomes to institutional and program effects, the need for 
a greater variety of criterion measures, and the problem of couching all 
outcomes in terms of measurable criteria. 

Several research directions and projects are suggested. First, it is 
obvious that an overview of the outcomes problem is necessary. Such a 
synthesis would include a review of cause and effect as well as descriptive 
outcome studies, an overview of the research on the outcome information 
needs of various kinds and levels of decision makers, and a review of eval- 
uation methodologies previously used to assess outcomes in higher education. 
Specific subsequent research projects might include: (1) an analysis of 
the inconsistencies among the value systems of the various decision-making 



V 



groups in higher education, (2) an analysis of the conflicting information 
needs of ' the different kinds and levels of decision makers, (3) a critical 
assessment of the propriety of general, overall evaluation methodologies, 
(4) the compilation of standardized instruments for the measurement of out- 
comes, (5) the development of lists of outcome indicators, and (6) the critical 
review and refinement of techniques for analyzing the relationships among 
environmental constraints, student and institutional resources, and educational 
outcomes and benefits. 
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PREFACE 



The financial difficulties of the institutions of higher education in 
the United States are well known. In 1971, the Carnegie Commission 
estimated that 61 percent of all institutions were either in financial 
difficulty or headed for it (Cheit, 1971). In a follow-up study it was 
found that some sectors of the higher education community had achieved some 
degree of financial stability. However, such stability certainly was 
not ubiquitous and was considered fragile where it had been achieved (Cheit, 
1973). It is not the purpose of this paper to debate the multitude and 
complexity of the causes of such problems. Rather, we suggest that planning 
in higher education (the complex process of establishing goals, evaluating 
alternatives, and allocating scarce resources for certain returns) can 
be improved by taking an "outcome-orientecl" (as opposed to a means-oriented) 
approach to the problem. Indeed, this change is being demanded by those who 
control the funds for higher education. 

In this paper, we do not seek to advance outcome-oriented planning as a 
cure-all. Rather, we have devoted a major part of this paper to discussion 
of the problems associated with this suggested approach. On the other 
hand, we do not view these problems as insurmountable. Accordingly, the 
final section of this paper discusses possible lines of future research 
which may facilitate the implementation of outcome-oriented planning. 
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INTRODUCTION 



Most college and university decision makers today are keenly aware of the 
pressures confronting higher education: the shifting demands concerning 
the number and kinds of educational goods and services, the increasing costs 
of providing these goods and services, and the increasing inability of higher 
education to deal effectively with its fiscal problems. Because of the mag- 
nitude of these pressures, which have affected virtually every institution 
in higher education, decision makers are recognizing the need for improved 
planning. They are also recognizing the need to delhonstrate that the outcomes 
of their programs are in keeping with the needs of different constituents. 
Prior to the emergence of these pressures, decision makers felt little need 
to obtain and use outcome information for planning and accountability. 
There are several reasons for this. 

First, historically, institutions have enjoyed almost unlimited moral "and 
financial support, and at the same time they have been afforded considerable 
autononiy in their operations. As a result, compared to private industry, 
higher education has not felt as much pressure for accountability in the use 
of resources. 

A second reason for this state of affairs is that decision makers in higher 
education— and in other areas as well --are often means-oriented in their 
planning and evaluation efforts rather than ends- or outcome-oriented. 
That is, they appear to react to problems and crises with more concern for 



ERIC 



what is wrong with the course of action being pursued—the means—and 
with less concern for what is to be achieved— the ends, 

A third reason is that past decisions have been based almost exclusively on 
historical input, costs and activity data, with little or no information about 
the outcomes and benefits derived from their institutions and programs. His- 
torically, money has been allocated largely on the k>asis of enrollments, number 
of graduates, and other quantitative measures related directly to costs rather 
than outcomes. However, these costs have seldom been evaluated in terms of 
achieving measurable goals or differential effectiveness. Consequently, 
it has simply been presumed that the outcomes produced by higher education 
programs have satisfactorily met individual and societal needs and expectations. 

Although most decision makers today recognize the necessity for a better 
understanding of the quantitative as well as the qualitative outcomes of 
higher education programs, they are quick to point out the problems involved. 
First, the specific desired outcomes of programs are difficult to identify 
and agree upon because the goals and values of individuals and institutions 
differ. Second, even when desired outcomes or objectives are identified and 
consensus is reached, few explicit criterion measures or indicators of program 
effectiveness are available. Finally, in cases where planners and decision 
makers have data about the outcomes of programs, they have much difficulty 
interpreting the data, since procedures for analyzing and using the information 
are not adequate. Such difficulty can be attributed to a host of factors: 
the complexity of higher education systems, the multiple and joint products 
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of programs, the unintended side effects of certain programs, the paucity of 
adequate statistical techniques for effective data analysis, and the lack of 
sufficient data bases for interprogram and interinstitutional comparison. 

ThG problems associated with obtaining and using outcome information for 
planning and accountability are legion; however, it is not the purpose of 
this paper to attempt a comprehensive discussion of all such problems. Instead, 
this paper attempts to describe a framework for an "outcome-oriented" approach 
to planning, a foundation for explaining and justifying the outcomes of insti- 
tutions and programs. Some of the obvious and not-so-obvious difficulties 
inherent in such an approach are discussed, and certain actions are suggested 
which may facilitate the implementation and acceptance of such an "outcome- 
oriented" approach to planning in higher education. 
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AN OUTCOME-ORIENTED APPROACH TO PLANNING IN HIGHER EDUCATION 

Planning is generally conceived to be a continuous process that involves 
the establishment of goals, the recognition and assessment of a number of 
alternative courses of action, the selection of those courses of action that 
will yield the most return on an investment, and the establishment of 
mechanisms for evaluation and feedback. This conception holds whether 
an individual planner or decision maker is means- or outcome-oriented. 
The unique characteristic of an outcome-oriented approach to planning is 
that an explicit understanding of the outcomes of a higher education in- 
stitution or program is emphasized in each phase of the planning procesis 
to facilitate both conwunication and comparison. 

The goal setting step emphasizes the translation of institutional and 
program goals, which often are broad and philosophical, into desired 
measurable outcomes terms. This is necessary to comprehend clearly the 
specific end results and benefits to be achieved. 

The program selection step emphasizes an understanding of the expected 
outcomes of each alternative program being considered. A result of this 
understanding is that the expected outcomes of each alternative program can 
be compared with the set of desired outcomes identified in goal setting; 
ideally, the alternative that most closely meets the desired outcomes can 
be identified. Of course, the selection decision will also be based on 
resource constraints. 
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The final step of the planning process, program evaluation , stresses 
understanding and measurement of the actual outcomes that result from 
implemented programs. Once the actual outcomes are determined, they 
can be compared with the desired and expected program outcomes * were 
determined earlier in the planning cycle. These comparisons help answer 
such questions as: Did the program meet the established goals? Did the 
program accomplish what it was e;cpected to do? Answers to these questions 
serve to raise further Inquiries about why certain outcomes did or did not 
occur. This information is valuable to Institutional planners and decision 
makers for demonstrating program success over one time period and making 
decisions in subsequent planning periods about the appropriateness of the 
established goals (desired outcomes) and the need for new programs or changes 
in continuing ones. 

This approach to planning based on an understanding of outcomes appears 
straightforward enough. However, there are certain prerequisites that 
must be met if such an approach is to bear fruit. 

The Need for a Comparable Outcomes Terminology 

A comparable outcomes terminology is essential to outcome-oriented planning, 
since the process continually emphasizes not only an understanding of the 
desired , expected , and actual outcomes of programs, but also a comparison. 
As a result, desired, expected, and actual program outcomes need to be 
stated in the same observable and/or measurable terms. This objective can 



5 



be accomplished by using descriptors and measures of the actual outcomes of 
institutions and programs as the comparable outcomes terminology. Then by 
conducting an "outcome translation" in the goal -setting and program selection 
phases of outcome-oriented planning, the desired and expected outcomes can 
be translated into actual outcome terms. The following discussion illustrates 
how an outcome translation process functions in each phase of the planning 
process. 

Ideally, goal setting begins by assessing the compelling national, state, 
or local needs in terms of the established philosophy and missions of the 
institution. Then, on the basis of the assessment of constituent needs, 
goalo should be formulated as clearly as possible. The goals should then 
be translated into specific, measurable outcome terms. In this last step 
of the goal -setting process, the outcome translation activity mentioned 
above is conducted to identify the specific actual program outcomes that 
reflect the formulated goals. The result of the outcome translation is 
that the actual outcomes identified as reflecting the established goals 
become the desired outcomes.* 

The program selection phase of planning has as its basic purpose the 
identification and comparison of alternative programs for achieving the 



*For an example of an attempt to construct inventories of educational 
outcomes and measures related to higher education programs see the 
appendices of Micek and Wallhaus (1973). 



desired outcomes specified in goal setting. To facilitate program selection, 
the goals of each alternative program under consideration must be translated 
into comparable outcome terms. This can be accomplished through the outcome 
translation process. The advantage is that the goals of each alternative 
are now stated in actual outcome terms, which are referred to as expected 
outcomes. These expected outcomes can then be hierarchically ordered (ranked 
or weighted) for each alternative and compared to a hierarchical ranking or 
weighting of the established desired outcomes. At this point, judgments must 
be made concerning the alternatives that best correspond to the ordered list 
of desired outcomes. Obviously, some alternatives will be eliminated immediately 
because they are economically or politically too costly. Ultimately, the 
process of elimination should narrow the number of alternatives to just a 
few. The final selection must be made subjectively. 

In the program evaluation phase of outcome-oriented planning, two major 
types of analysis are carried out. The first type concerns an analysis 
of the measured actual outcomes of the program. Obviously, this analysis 
is facilitated by the fact that the desired and expected outcomes have 
been stated in the same terms as the actual outcomes. The second type of 
analysis focuses on why a discrepancy occurred or did not occur. Once these 
analyses have been made, information from this part of the planning process 
can be fed back into future planning efforts, and judgments can be made 
concerning the continuation, modification, or replacement of programs being 
implemented as well as the selection of new alternatives for satisfying new 
goals and objectives. 
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The Comppnents of a Higher Education "System" in a Planning Context 

Another key requisite for achieving the benefits of planning based on outcomes 
information is the consideration of the interrelationships among the components 
of the particular higher education institution or program being studied. Under- 
standing these components is important, since on the one hand the components 
shape and condition the planning efforts, while on the other, planning attempts 
to shape the components to a greater or lesser extent. In addition to the 
outcomes component, a conceptual model of an institution or program should 
at least include: the inputs, the educational process itself, and the institu- 
tional environment in which the process occurs. Furthermore, the inputs could 
be conceptualized as consisting of both institutional resources (physical, 
financial, and facu Ity/ staff ) and student resources (intellectual, attitudinal, 
and financial ) . 

The interrelationships among these components are complex. The inputs affect 
the environment in many ways (e.g., the construction of new buildings, the 
sex mixture on campus) and also affect the educational process (that is, the 
number of students, for example, might dictate what type of instructional 
mode would be most effective). The process and the environment also interact 
(for example, the educational process possibly may attract certain types of 
students and faculty while the environment places constraints on the process). 
In addition, the outcomes of the system can affect the inputs in future time 
periods (such as the attraction of a different kind of faculty, or the 
achievement of greater financial stability). 
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The relationship between planning and any conceptual model of a higher 
education "system'* (e.g., an institution, program, course) is clear 
if one realizes that an essential phase in planning is the choice of a 
single program from among a numberof alternatives. In order to make a 
choice the decision maker must have a way of differentiating among programs. 
Programs can differ on the inputs required, the process by which certain 
outcomes will be achieved, and the institutional environment necessary for 
the implementation of the program* In short, programs can differ with 
respect to each of the components comprising a conceptual model of 
higher education, and these differences must be considered in outcome-oriented 
planning. 

Relationship of Outcome -Oriented Planning to. Accountability 

In the present discussion of the requisites for implementing this outcome- 
oriented approach to planning, it is Important to discuss its relationship 
to the demand for accountability. Outcome-oriented planning has a major 
advantage in that the outcome information obtained for internal planning 
purposes may help provide part of the Information needed by decision makers 
to explain and justify their programs to various constituents. An important 
consideration, however, is that the outcome information preferred by certain 
' constituents for their planning decisions will often differ from that pre- 
ferred by others. The main reason for this is that the perspectives and 
decision situations of different kinds of decision makers vary. For example, 
college administrators and state legislators may both value information about 
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the "specialized knowledge" of students. However, the administrator might 
view scores on special standardized tests as the best measure of this outcome 
variable, while the legislator might consider the job performance of former 
students in their major field of study as the best measure of this variable. 

PROBLEMS IN MEASURING HIGHER EDUCATION OUTCOMES* 

While some of the requisites and advantages of an outcome-oriented approach 
to planning have been touched on, many difficulties hinder implementation. 
In particular, there are the problems associated with obtaining reliable 
information about the differential outcomes or impacts of higher education 
institutions and programs. This section of the paper discusses several of 
these problems in some detail, since solutions must be Identified if outcome- 
ortented planning (and educational accountability) is going to be implemented 
effectively. 

The Problem of Defining and Assessing Institutional Goals 

Earlier it was argued that any evaluation of an institution's effectiveness 
must be based on an institution's goals Und preferably goals that have been 



*The authors are* indebted to Dr. Rodney T. Hartnett of Educational Testing 
Service for his contribution to this section of the paper, which paraphrases 
part of a publication by Dr. Hartnett entitled Accountability in Higher 
Education: Some Problems in the Assessment of College Impacts . It should 
be noted that although Dr. Hartnett 's paper has been published by the College 
Entrance Examination Board, it was originally written for inclusion in this 
paper. 
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translated into desired outcome terras). The problem, of course, is that 
too few institutions have really considered what their goals are, and those 
that have often find that the various members of the college community dis- 
agree over what the purposes of the institution should be. For example, studies 
by Gross and Grarabsch 0968) and the Danforth Foundation (1970), which surveyed 
faculty and administrators, reported marked differences between the pe»"ceived 
institutional goals and the Institutional goals that would be preferred. 

At least one recent study attempts to construct an inventory of goals for 
institutions of higher education (Peterson, 1971). A preliminary form of 
one such inventory contains 100 statements of Institutional goals (for example, 
"to help students develop the ablility to speak and write effectively," 
"to strengthen the religious faith of students," "to assist in efforts to 
achieve and maintain world peace,") to which the respondents— students, 
faculty, administrators, alumni, trustees, members of the immediate com- 
munity, or whatever— indicate the extent to which they feel each statement 
Is and should be a goal of the institution. Such an approach makes several 
things possible. First, while it may be true that divergent groups will 
never see eye to eye on the major purposes of higher education institutions, 
it will at least be possible to quantify the extent of their disagreement 
and account for it in subsequent studies. Second, the technique provides 
a measure of discrepancy between what each of the groups thinks is and 
should be highly valued in academia- Third, if an investigation were carried 
out on a national sample of institutions, it would be possible to find how 
a particular institution's emphasis (goals) differed from the national 
sample as well as from similar institutions (such as junior colleges, large 
universities, etc.)- 
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The Problem of Emphasizing. Measurable Objectives in Assessing Higher 
Educational Outcomes ^ 

Most statements of educational goals are too general in nature to permit precise 
assessment of whether they have been achieved. How does one determine whether or 
not an institution has "prepared students for the duties and responsibilities of 
citizenship," or "enables students to develop a set of principles to guide their 
behavior," or any of a whole series of similar statements that might be found in 
college catalogues? It was concerns such as these that led to an emphasis on the 
identification of measurable objectives in education (e.g., the specification of 
"behavioral objectives" that are operational definitions or statements of specific 
educational objectives in terms of changed student behavior). Such objectives 
lend themselves nicely to precise observation and measurement. However, such 
precision can be restrictive in that highly desirable educational outcomes not 
susceptible to such measurements are simply omitted. 

It would be unfortunate indeed if institutions were evaluated solely in terms 
of how well their students performed on measures of objectives that were 
employed primarily because they could be measured! That would be the proverbial 
tail wagging the dog. Cronbach. (1969) has pointed out that specific behavior 
and other measurable or observable properties can and should be employed as 
indicators of constructs (such as self-confidence or scientific attitude ) but 
not as the definers of those constructs. Cronbach argues that constructs 
ought to be the crucial aspect of the evaluation process, where constructs * 
refer to a network of relations or characteristics, but not specific Incidents 
of behavior. Cronbach goes on to say that "The operational ists who want to 
equate each construct with *one indicator' .. .are advocating that we restrict 



descriptions to statements of tasks performed or behavior exhibited and are 
rejecting construct interpretations. . .The writers on curriculum and eval- 
uation who insist that objectives be 'defined in terms of behavior' are 
taking an ul traoperational ist position, though they have not offered a 
scholarly philosophical analysis of the issue." 

To use as definitions of educational goals— at any level of education— only 
those outcomes that can be measured will almost certainly result in a neat 
list of narrow and unimportant educational outcomes. Not^ to attempt to state 
educational objectives in some measurable way tempts educators to rely on 
the sort of rhetoric that has characterized college catalogues for many years. 
The dileima is a struggle between what Melvin Tumin (1970:68) calls "trivial 
precision and apparently rich ambiguity," and it is imperative that institutional 
administrators and faculty members get together with the educational evaluators 
or "accountants" and attempt to strike a better balance between these two 
extremes. 

The Lack of Variance Problem 

Proponents of outcome -oriented planning and educational accountability tend 
to favor a "value added" concept. That is, ir '*;itutions and programs should 
be judged not by their outputs alone, but by their outputs relative to their 
"inputs," The students' final "standing" with regard to various characteristics 
would not be as important as their changes (usually gains) during the college 
years, A rather typical point of view is the following: "What has the 
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student attained in relation to his capability at the starting point? This 
concept approximates educational value-added > . . . According to this view, 
an educational process which moved the student from the lowest quartile of 
high school achievement to the second quartile of college-graduate achievement 
would be accomplishing something tremendous, whereas the college which accepted 
students only from the top decile of high school achievement and delivered them 
into the top decile of college achievement would be doing relatively much less" 
(Balderston, 197Q:14). 

Such a view— and again, it should be emphasized that it is a view widely 
held— makes the assumption that educational institutions are potentially 
very powerful change agents, capable of having considerable impact on 
both the cognitive and noncognitive attributes of those who pass through 
th^ir doors. It is further assumed that colleges differ widely in the amount 
of impact they have. The accuracy of such a view is questionable, however, 
since most of the evidence suggests that educational institutions differ 
very little in terras of the amount of impact they have on their students 
after controls are made for general mental ability, socioeconomic status, 
and other important background factors outside the purview of the formal 
educational institution. For example, numerous proponents of the "value 
added" concept in educational accountability argue that one good indicator 
of institutional quality would be the students' standing on standardized 
tests of educational "attainment," after controls have been made for 
educational aptitude at the time of en^y into college. Very often 
specific suggestions are made for use of one of the national college 
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admissions tests (the Scholastic Aptitude Test of the College Entrance 
Examination Board or the tests of the American College Testing Program) 
as the input measure, and scores on one of the Area Tests of the Graduate 
Record Examinations (GRE) as the output measure,* At first blush, such 
an approach seems quite sensible. The problem, however, is that the 
correlation between college means on these measures is so high (often 
in the .90s), that there is generally very little variance left that the 
schools can influence. 

Part of the difficulty in discovering differential cognitive impact of 
educational institutions may be attributable to the overemphasis on stand- 
ardized tests. Given the nature of most tests of cognitive attributes used 
in such research, it probably shouldn't be too surprising that they do not 
turn up large educational differences. These tests are almost always 
constructed so as to be widely appropriate and sufficiently general in 
nature to ensure their appropriateness for many educational experiences. 
Yet herein lies part of the evaluative problem. Tests designed to be broadly 
applicable may well be too general to measure the specific outcomes of 
educational experiences at a local level. Educational evaluators may have 
to turn, instead, to achievement examinations geared especially to syllabi 
used in specific college courses if they are to turn up indices of college 
effects. Such a procedure makes it difficult, however, to conduct inter- 
Institutional comparisons, often felt to be the central and most important 



♦Technically, the Graduate Record Examinations now refer exclusively to 
the aptitude and achievement measures (Advanced Tests) used for graduate 
school admission. The tests formerly known as the GRE Area Tests are 
now part of ETS's new Undergraduate Program for Counseling and Evaluation 
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feature of educational accounting systems. Thus, we are back to the problems 
suggested earlier; measures of a general nature yield little or no interinsti- 
tutional variation, while measures geared to the program of a specific department 
or institution do not allow for multicollege comparisons. Yet the interinstitu- 
tional comparisons are useless If they fail to reveal meaningful differences, 
and so specifically designed instruments for measuring outcomes may be the only 
reasonable solution. 

The Need for New Outcome Measures 

Three issues are, particularly relevant to the need for a greater variety of 
outcojne measures. First, the measurement of outcomes should not be limited 
to the intellectual domain but should extend to the affective domain of the 
students. This includes personality adjustments and changes, social develop- 
ment, cultural interests, vocational development, philosophical orientation, 
etc. Feldman and Newcorab (1969) reviewed the literature related to college 
impact in a number of these areas. One striking conclusion derived from 
their work is that impact studies have tended to focus on those aspects of 
the college experience for which some standard measure or test exists. One 
problem with this practice is that the research community as a whole can become 
dependent on a single instrument which, with time, can become outdated or 
irrelevant. Also, generalization from specific studies to wider populations 
may be inappropriate. 

The second issue relates to the fact that the college experience is future- 
oriented. In other words, it is usually the goal of an institution to prepare 



its students for their future life. Some educational philosophers feel thii^ 
is a societal requirement that can validly be placed on higher education. In 
addition, students tend to view college preparation as a facilitator of upward 
mobility. This function of higher education has two implications. Data for 
variables that measure long-term effects, vocational as well as, personal, can 
be collected. Knowledge of these effects is necessary if colleges are to account 
properly for their money since long-range effects are stated as goals. Second, 
it will be necessary to develop individual -centered outcome measures as well as 
institutional -aggregated measures. If this is done, educational counselors 
will be better equipped to assess' what an individual will lose or gain by 
making a decision to attend or not to attend college. 

The third issue deals with the accelerating trend toward universal higher 
education. Here higher education is used to mean some formal training past 
the high school level. An increasing number of persons today continue their 
education beyond high school, and they are demanding different kinds of 
educational experiences. This has resulted in open admissions policies, 
community colleges, vocational schools, and a number of other programs 
making higher education more available to everyone. Thus, outcome measures 
for educational experiences considerably different from those of the tradi- 
tional liberal arts colleges or large multiversities need to be developed. 
Some efforts have been made in this area, with emphasis on community colleges 
(Richards et al . , -.965). Second, the heterogeneity of the new student bodies 
must be considered. In the past decade it has come to be well known that 
student inputs determine to a high degree the outcomes of educational programs. 



Given that the diversity of student bodies is currently increasing (Cross, 1970), 
we can expect a greater variance in outcome measures, variance that may necessi- 
tate the adoption of new indicators of those outcomes • 

NEEDED IMPLEMENTATION AND RESEARCH DIRECTIONS 

It seems unlikely that higher education alone among social institutions will 
escape the pervasive use of some form of cost-effective planning and manage- 
ment. An overriding concern is that current pressures will lead to implementa- 
tion of an irrelevant and inappropriate system. The fear is that this system 
will persist far into the future simply because it has been established and 
relieves just enough pressure to forestall improvement or substitution of an 
appropriate system. Such a concern does not remove, however, the necessity of 
beginning' as soon as possible activities aimed at helping decision makers obtain 
and use outcome information for planning and accountability purposes. 

One major area of activity should focus on what can be implemented in the 
immediate future, since the need for outcome information is urgent. Identifi- 
cation of the critical outcome information needs of different decision makers 
(faculty, students, administrators, legislators, etc.) is basic for determining 
appropriate measures of the outcomes and establishing the conventions for 
data analysis and information dissemination. A proposed project would serve to 
inform decision makers at different levels of higher education about their 
common and unique information needs. The results of such a project not only 
would help various decision makers understand the outcome information that 
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others value, but also would help set priorities concerning what outcome in- 
formation should be collected, what measurement tools should be secured, and what 
will be the communication and dissemination requisites. 

Another implementation-oriented effort that could have immediate payoff involves 
identifying what outcome information already is being collected and how the 
information can be processed. For example, the American Council on Education's 
Cooperative Research Program collects a wealth of data on students, as do major 
testing organizations like the American College Testing Proqram and Educational 
Testing Service. How these data banks might be tapped and how such information 
might be used for making better decisions are questions for which it would be 
valuable to have answers. 

Identifying available measurement tools that are appropriate for conducting 
empirical studies of differential outcomes among programs, departments, and 
institutions could also help in the immediate future. It would be useful to 
summarize these instruments in a form that would be meaningful and readily 
accessible to all decision makers and institutional researchers in higher ed- 
ucation. 

A second major area of activity involves the research and development efforts 
needed to obtain and use outcome information. Basic to the identification of 
such efforts would be an integrative overview of the problem area. Such an 
integration would involve theorists and researchers from a variety of disciplines 
As an understanding of the outcomes problem evolves, related research and 
development projects might be Initiated concurrently. 



One concurrent research effort could be directed toward the following question: 
"Can there be a general evaluation methodology that would be adaptable to most 
situations, or should evaluators construct customized evaluation schemes for each 
setting?" There have been several attempts to construct "models" of evaluation. 
Among the best known are Stake's 0967) countenance model, Stufflebeam's (1971) 
CIPP Model, and Provus's 0971} discrepancy model. Research needs to focus on 
whether these models could be advantageous for evaluation. 

Vaizey 0970) and other researchers concerned with the outcome problem in higher 
education strongly advocate cohort longitudinal studies of the impact of college 
on students. These are expensive and, furthermore, entail long periods of time, 
which runs counter to the need for immediate results. However, careful consider- 
ation of the problem indicates such studies would in the long run prove more use- 
ful than any other type. 

While there is a critical lack of measures appropriate for measuring the outcomes 
of higher education, techniques for data analysis also deserve critical review. 
Standard techniques that seek optimization are not useful for situations where 
criteria are undefined, multiple, or conflicting. Basic research on the develop- 
ment of effective statistical techniques is needed, as is a clarification of 
the shortcomings of present methodologies. 

It is easy to suggest implementation and research directions that might be 
pursued. It is clear, however, that no one institution or organization will 
be able to carry out all of the needed immediate and long-range implementation 
and research studies. As a result, a systematic coordination of both practitioners 



and researchers should be undertaken. ysuch a coordinated approach could make 
expertise available when and where it is required for stimulating certain imple- 
mentation and research efforts, synthesizing others, and promoting mutually sup- 
portive communications among practitioners and researchers. It could encourage 
the dissemination of results in such a way as to minimize misuse of incomplete 
information and provitle timely feedback for improved capabilities. Finally, such 
an approach could result in the consideration and consolidation of the many view- 
points relative to the use of outcome Information and could assure that all sectors 
of the higher education community are represented. This would be necessary to 
gain the credibility and political viability that is essential for the effort to 
achieve success* 

Conclusion 

This paper has discussed a framework for an outcome-oriented approach to planning, 
some of the inherent difficulties in measuring higher education outcomes, and cer- 
tain implementation and research directions that are indicated if outcome infor- 
mation is going to play a major role in planning. 

Whether or not an outcome-oriented approach to planning will prove to be a viable 
approach or an impossibility remains to be seen. Dr. Fred Balderston may have 
summed up the situation best in these words: 

We have bumped hard into the question of outputs and their measurements 
because, among other things, we are seeking how to link the resources 
used to the results achieved— in other words, to link inputs with 
outputs. It turns out that in the long history of concern about the 



ERIC 



21 



processes and activities of education, we have achieved only a very 
imperfect grasp of the nature of its results. Now we are having to 
tackle the problems of ouput definition and measurement under forced 
draft, for higher education has come to the front of public attention 
both as a major social problem and as a major contributor to social 
change and economic development. . . . The job we have to do is urgent, 
important, and controversial. If we had time, we might do well to sympa- 
thize with ourselves for taking it on (Balderston, 1970: 11). 



It is important to point out that in 1970, when this statement was made, it 
was considered by many to be a. long-range prophecy. Today, however, the need 
for and urgency of using information about the outcomes of higher education 
programs and institutions as a primary basis for planning and management is 
understood and recognized as a compelling reality. 
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